Gesture detection device for detecting hovering and click

ABSTRACT

There is provided a gesture detection device including two linear image sensor arrays and a processing unit. The processing unit is configured to compare sizes of pointer images in the image frames captured by the two linear image sensor arrays in the same period or different periods so as to identify a click event.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan PatentApplication Serial Number 102103852, filed on Jan. 31, 2013, the fulldisclosure of which is incorporated herein by reference.

BACKGROUND

1. Field of the Disclosure

This disclosure generally relates to a human-machine interaction systemand, more particularly, to an optical gesture detection device capableof performing the hover detection and the click detection.

2. Description of the Related Art

As the hand gesture control is operated according to human instinct suchthat it is widely applied to home appliances and portable electronicdevices. The user can implement the specific function only by givingpredetermined postures. In this way, more operating functions can beprovided in addition to the traditional functions provided by pressingbuttons.

The hand gesture control can be divided into the touch control and thehover control according to the operating method thereof, wherein thetouch control system generally employs a touch panel such that the usercan perform the slide control and the click control on the touch panel.However, the touch panel itself has a high manufacturing cost.

The hover control system generally utilizes an image sensor to detectthe hovering gesture, i.e. the optical gesture control system. Forexample, a camera can be used to capture two-dimensional image framesand the move information can be identified according to the positionvariation of finger in the successive two-dimensional image frames.However, it is difficult for the hover control system to detect theclick event, which is also an important issue in the field.

Accordingly, the present disclosure further provides a gesture detectiondevice that may detect both the hovering operation and click operationby using at least two image sensors and eliminate the interferencecaused by ambient light.

SUMMARY

The present disclosure provides a gesture detection device that employsthe linear image sensor array so as to reduce the cost and increase thenumber of detectable images per second.

The present disclosure further provides a gesture detection device thatmay eliminate the interference of ambient light by calculating thedifferential image thereby improving the identification accuracy.

The present disclosure provides a gesture detection device including adetection surface, a first image sensor, a second image sensor and aprocessing unit. The detection surface has a normal direction. The firstimage sensor is disposed on the detection surface and captures a firstimage along the normal direction. The second image sensor is disposed onthe detection surface offsetting a predetermined distance from the firstimage sensor along a Z-axis and captures a second image along the normaldirection. The processing unit is configured to calculate an X-axiscoordinate according to a position of a pointer image in the first imageor a pointer image in the second image, calculate a Y-axis coordinateaccording to a disparity of the pointer images in the first image andthe second image, and identify a click event of the Z-axis when a sizedifference of the pointer images between the first image and the secondimage is larger than or equal to a difference threshold.

The present disclosure further provides a gesture detection deviceincluding a detection surface, a first image sensor, a second imagesensor and a processing unit. The detection surface has a normaldirection. The first image sensor is disposed on the detection surfaceand captures first images along the normal direction. The second imagesensor is disposed on the detection surface parallel to the first imagesensor and captures second images along the normal direction. Theprocessing unit is configured to calculate an X-axis coordinateaccording to a position of a pointer image in the first images or apointer image in the second images, calculate a Y-axis coordinateaccording to a disparity of the pointer images in the first images andthe second images, and identify a click event when a size variation ofthe pointer images between the first images captured at different timesand/or between the second images captured at different times is largerthan or equal to a variation threshold.

The present disclosure further provides a gesture detection deviceincluding a detection surface, a first image sensor, a second imagesensor, a third image sensor and a processing unit. The detectionsurface has a normal direction. The first image sensor is disposed onthe detection surface and captures a first image along the normaldirection. The second image sensor is disposed on the detection surfaceparallel to the first image sensor and captures a second image along thenormal direction. The third image sensor is disposed on the detectionsurface offsetting a predetermined distance from the first image sensorand the second image sensor along a Z-axis and captures a third imagealong the normal direction. The processing unit is configured tocalculate a two-dimensional coordinate according to the first image andthe second image, and identify a click event of the Z-axis when a sizedifference between a pointer image in the first image and a pointerimage in the third image or between a pointer image in the second imageand the pointer image in the third image is larger than or equal to adifference threshold.

The present disclosure further provides a gesture detection deviceincluding a detection surface, a first image sensor, a second imagesensor and a processing unit. The detection surface has a normaldirection. The first image sensor is disposed on the detection surfaceand captures a first image along a direction deviating a predeterminedangle from the normal direction. The second image sensor is disposed onthe detection surface offsetting a predetermined distance from the firstimage sensor along a Z-axis and captures a second image along adirection deviating a negative predetermined angle from the normaldirection. The processing unit is configured to calculate atwo-dimensional coordinate according to positions of a pointer image inthe first image and a pointer image in the second image, and identify aclick event of the Z-axis when a size difference of the pointer imagesbetween the first image and the second image is larger than or equal toa difference threshold.

The present disclosure further provides a gesture detection deviceincluding a detection surface, a first image sensor, a second imagesensor and a processing unit. The detection surface has a normaldirection. The first image sensor is disposed on the detection surfaceand captures first images along a direction deviating a predeterminedangle from the normal direction. The second image sensor is disposed onthe detection surface parallel to the first image sensor and capturessecond images along a direction deviating a negative predetermined anglefrom the normal direction. The processing unit is configured tocalculate a two-dimensional coordinate according to positions of apointer image in the first images and a pointer image in the secondimages, and identify a click event when a size variation of the pointerimages between the first images captured at different times and/orbetween the second images captured at different times is larger than orequal to a variation threshold.

The present disclosure further provides a gesture detection deviceincluding a detection surface, a first image sensor, a second imagesensor, a third image sensor and a processing unit. The detectionsurface has a normal direction. The first image sensor is disposed onthe detection surface and captures a first image along a directiondeviating a predetermined angle from the normal direction. The secondimage sensor is disposed on the detection surface parallel to the firstimage sensor and captures a second image along a direction deviating anegative predetermined angle from the normal direction. The third imagesensor is disposed on the detection surface offsetting a predetermineddistance from the first image sensor and the second image sensor along aZ-axis and captures a third image along the normal direction. Theprocessing unit is configured to calculate a two-dimensional coordinateaccording to the first image and the second image, and identify a clickevent of the Z-axis when a size difference between a pointer image inthe first image and a pointer image in the third image or between apointer image in the second image and the pointer image in the thirdimage is larger than or equal to a difference threshold.

In one aspect, when sizes of the pointer images in the first image andthe second image are substantially identical and larger than or equal toa predetermined size, it means that the pointer images are not fingerimages and thus the processing unit does not calculate the hovercoordinate and does not identify the click event.

In one aspect, the gesture detection device may further include aninfrared light source configured to illuminate field of views of theimage sensors, wherein the infrared light source may emit lightalternatively in a first brightness value and a second brightness value.The image sensors may respectively capture a bright image correspondingto the first brightness value and a dark image corresponding to thesecond brightness value. The processing unit may calculate adifferential image between the bright image and the dark image capturedby each image sensor so as to eliminate the interference of ambientlight, wherein the first brightness value may be larger than the secondbrightness value, and the second brightness value may be zero or nonzerobrightness.

In one aspect, the image sensors include a linear image sensor array,e.g. having 750 pixels in length and 1-6 pixels in width, to form alinear field of view.

The gesture detection device according to the embodiment of the presentdisclosure may be formed as an independent detection device and coupledto the electronic device having a screen, e.g. a smart TV, throughwireless communication (for example, but not limited to, the bluetoothcommunication or microwave communication) or wired interface (forexample, but not limited to, the USB interface). In addition, thegesture detection device according to the embodiment of the presentdisclosure may be directly integrated in the portable electronic devicehaving a screen, e.g. notebook computers, personal digital assistancesor cell phones.

In the gesture detection device according to the embodiment of thepresent disclosure, the image sensors may be arranged to capture imageframes toward a direction substantially along a normal line or adirection inwardly deviating a predetermined angle from the normal lineaccording to the distance between the image sensors such that a pointerimage may simultaneously appear in field of views of all the imagesensors.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, advantages, and novel features of the present disclosurewill become more apparent from the following detailed description whentaken in conjunction with the accompanying drawings.

FIG. 1 shows a schematic block diagram of the gesture detection deviceaccording to an embodiment of the present disclosure.

FIGS. 2A and 2B show schematic diagrams of the gesture detection deviceaccording to a first embodiment of the present disclosure.

FIGS. 3A and 3B show schematic diagrams of the gesture detection deviceaccording to a second embodiment of the present disclosure.

FIGS. 4A-4C show schematic diagrams of an electronic device incooperation with the gesture detection device according to theembodiment of the present disclosure.

FIG. 5 shows an upper view of a first aspect of the gesture detectiondevice according to the embodiment of the present disclosure.

FIG. 6 shows an upper view of a second aspect of the gesture detectiondevice according to the embodiment of the present disclosure.

FIGS. 7A-7C show upper views of a third aspect of the gesture detectiondevice according to the embodiment of the present disclosure.

FIG. 8 shows a schematic diagram of calculating an X-axis coordinate inthe gesture detection device according to the embodiment of the presentdisclosure.

FIG. 9 shows a schematic diagram of calculating a Y-axis coordinate inthe gesture detection device according to the embodiment of the presentdisclosure.

FIGS. 10A and 10B show schematic diagrams of identifying a click eventin the gesture detection device according to the embodiment of thepresent disclosure.

FIG. 11 shows a schematic diagram of calculating the two-dimensionalcoordinate of a hovering pointer in the gesture detection deviceaccording to the embodiment of the present disclosure.

FIG. 12 shows a schematic diagram of the lighting of light source andthe image capturing of image sensor in the gesture detection deviceaccording to the embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT

It should be noted that, wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.

Referring to FIG. 1, it shows a schematic block diagram of the gesturedetection device 1 according to an embodiment of the present disclosure.The gesture detection device 1 includes at least two image sensors, aprocessing unit 12 and a light source 13. The number of the imagesensors may be determined according to different applications. In oneembodiment, the gesture detection device 1 may include a first imagesensor (S₁) 111 and a second image sensor (S₂) 112. In anotherembodiment, the gesture detection device 1 may include a first imagesensor (S₁) 111, a second image sensor (S₂) 112 and a third image sensor(S₃) 113. The processing unit 12 may be a digital signal processor andconfigured to receive image frames outputted by the at least two imagesensors for post-processing to accordingly identify a two-dimensionalcoordinate of the hovering operation and a click event. The light source13 is configured to illuminate field of views of the at least two imagesensors and preferably emit invisible light, e.g. infrared light. In oneembodiment, in order to eliminate the interference from ambient light,the light source 13 may emit light with different brightness valuescorresponding to the image capturing of the at least two image sensors(described later).

In addition, in order to decrease the cost and increase the number ofdetectable images per second, the at least two image sensors may beactive image sensors, e.g. CMOS image sensors, and respectively mayinclude a linear image sensor array, wherein the term “linear” herein isreferred to the image sensor array having a length much larger than awidth thereof, e.g. 750 pixels in length and 1-6 pixels in width so asto form a linear field of view. In addition, the light source 13 may beformed by arranging a plurality of active light sources or may be asingle active light source (e.g. LED) as long as enough brightness canbe provided thereby.

Referring to FIGS. 2A and 2B, they show schematic diagrams of thegesture detection device 1 according to a first embodiment of thepresent disclosure, which includes a detection surface 10, and a normaldirection of the detection surface 10 is indicated by “n”. The firstimage sensor 111 and the second image sensor 112 are disposed on thedetection surface 10 and configured to capture image framessubstantially toward the normal direction “n”; that is, the first imagesensor 111 and the second image sensor 112 are arranged in the way sothat sensing surfaces thereof are substantially parallel to thedetection surface 10. In one embodiment, the first image sensor 111, thesecond image sensor 112 and the light source 13 may be disposed at aninner side of the detection surface 10 as shown in FIG. 2A. In anotherembodiment, the first image sensor 111, the second image sensor 112 andthe light source 13 may be disposed on the detection surface 10 as shownin FIG. 2B. In this embodiment, center lines of the field of views ofthe first image sensor 111 and the second image sensor 112 may have anincluded angle, e.g. 0-20 degrees, with respect to the normal direction“n” so as to use the hovering operation algorithm of the firstembodiment of the present disclosure mentioned below to calculate thehover coordinate, wherein the included angle preferably rotate toward acenter position between the two image sensors.

Referring to FIGS. 3A and 3B, they show schematic diagrams of thegesture detection device 1 according to a second embodiment of thepresent disclosure, which includes a detection surface 10, and a normaldirection of the detection surface 10 is indicated by “n”. The firstimage sensor 111 and the second image sensor 112 are disposed on thedetection surface 10 and configured to capture image frames toward adirection deviating a predetermined angle from the normal direction “n”;that is, the first image sensor 111 and the second image sensor 112 arearranged in the way so that sensing surfaces thereof rotate with respectto the detection surface 10 by the predetermined angle. In oneembodiment, the first image sensor 111, the second image sensor 112 andthe light source 13 may be disposed at an inner side of the detectionsurface 10 as shown in FIG. 3A. In another embodiment, the first imagesensor 111, the second image sensor 112 and the light source 13 may bedisposed on the detection surface 10 as shown in FIG. 3B. In thisembodiment, center lines of the field of views of the first image sensor111 and the second image sensor 112 preferably have a predeterminedangle, e.g. θ=30-60 degrees, with respect to the normal direction “n” soas to use the hovering operation algorithm of the second embodiment ofthe present disclosure mentioned below to calculate the hovercoordinate, wherein the predetermined angle preferably rotate toward acenter position between the two image sensors. As shown in FIGS. 3A and3B, the first image sensor 111 rotates an angle of +θ whereas the secondimage sensor 112 rotates an angle of −θ, wherein the symbols±hereinindicate opposite rotation directions.

It is appreciated that when the image sensors are disposed at an innerside of the detection surface 10, preferably at least a part of thedetection surface 10 is transparent to the light emitted by the lightsource 13; for example, a transparent cover may be used to cover thereonso as to protect elements inside the detection surface 10. When theimage sensors and the light source are disposed on the detection surface10, a transparent cover may also be used to cover thereon so as toprotect the elements.

The gesture detection device 1 according to the embodiment of thepresent disclosure may be integrated with a portable electronic device,e.g. a notebook computer, a tablet computer, a cell phone or a personaldigital assistance and configured to detect a hover coordinate of apointer (e.g. a finger) and a click event as shown in FIG. 4A. Inaddition, the gesture detection device 1 may be formed as an independentdevice and wired (as shown in FIG. 4B) or wirelessly (as shown in FIG.4C) coupled to an electronic device 9 having a screen, e.g. a computersystem, a television or a projection system. It is appreciated that thegesture detection device 1 may include a transmission unit configured towired or wirelessly transmit the detected results to the electronicdevice 9. In one embodiment, the hover coordinate detected by thegesture detection device 1 may be used to control the motion of a cursor91, and the click event (e.g. a single click or multi-click) may be usedto activate an application associated with a selected icon.

Referring to FIGS. 2A, 2B and 5, FIG. 5 shows an upper view of a firstaspect of the gesture detection device 1 according to the firstembodiment of the present disclosure. In this aspect, the first imagesensor 111 is disposed on the detection surface 10 and configured tocapture a first image IF₁ substantially along the normal direction “n”;the second image sensor 112 is disposed on the detection surface 10offsetting a predetermined distance D from the first image sensor 111along a Z-axis and configured to capture a second image IF₂substantially along the normal direction “n”; and the processing unit 12is configured to calculate an X-axis coordinate according to a positionof a pointer image in the first image IF₁ or the second image IF₂,calculate a Y-axis coordinate according to a disparity of pointer imagesin the first image IF₁ and the second image IF₂, and identify a clickevent along the Z-axis when a size difference of the pointer imagesbetween the first image IF₁ and the second image IF₂ is larger than orequal to a difference threshold. Methods of identifying the X-axiscoordinate, the Y-axis coordinate and the click event are describedbelow.

Referring to FIG. 8, when a pointer image 1p is contained in the firstimage IF₁ or the second image IF₂, the pointer image Ip may cover aplurality of pixels. The processing unit 12 may calculate a position ofthe pointer image Ip (e.g. a center position or a gravity centerposition of the pointer image Ip) to be served as the X-axis coordinate.For example in one embodiment, it is assumed that the image sensorincludes a linear image sensor array having a size of 750×1 such thatevery pixel position within an operable range 0-Rx (e.g. 0-750 herein)may be set to respectively correspond to one X-axis coordinate.Therefore, the processing unit 12 may calculate a position of thepointer image Ip within the operable range 0-Rx to be served as theX-axis coordinate.

If the X-axis coordinate is configured to be mapped to a projectionscreen, the processing unit 12 may further map the X-axis coordinateonto a correct position of the projection screen according to a ratiorelationship between an X-axis resolution of the projection screen andthe operable range, wherein the mapping method is well known and thusdetails thereof are not described herein. It is appreciated that in thisembodiment, the operable range may include only the central part of thelinear image sensor array (for example, but not limited to, from 50pixels to 700 pixels) without including pixels at two edges so as toavoid miscalculation of the position when the pointer image Ip locatesat the two edges.

Referring to FIG. 9, when the first image IF₁ and the second image IF₂respectively contain a pointer image, the processing unit 12respectively calculates a position (e.g. a center position or a gravitycenter position) of a first pointer image Ip₁ in the first image IF₁ anda second pointer image Ip₂ in the second image IF₂. Then, the processingunit 12 calculates a position difference between positions of the firstpointer image Ip₁ and the second pointer image to be served as adisparity. For example, when a pointer is farther from the imagesensors, the first pointer image Ip₁ and the second pointer image Ip₂are closer to each other; whereas when the pointer is closer to theimage sensors, the first pointer image Ip₁′ and the second pointer imageIp₂′ are farther from each other. Therefore, every disparity may be setto correspond to one Y-axis coordinate, and the processing unit 12 maycalculate the disparity to be served as the Y-axis coordinate.

It is appreciated that the gesture detection device 1 according to theembodiment of the present disclosure may include a storage unitconfigured to save the relation between the X-axis coordinate versus theposition of the pointer image Ip and the relation between the Y-axiscoordinate versus the disparity.

Referring to FIG. 10A, in this aspect the first image sensor 111 and thesecond image sensor 112 are configured to detect a hovering operation ofa finger and a click event. For example, at time t₁, if the firstpointer image Ip₁ in the first image IF₁ and the second pointer imageIp₂ in the second image IF₂ have substantially identical areas orwidths, it means that it is the finger of user between the first imagesensor 111 and the second image 112, and the user is performing thehovering operation now. In hovering operation the X-axis coordinate andY-axis coordinate may be calculated by the method of FIGS. 8 and 9 to beserved as the two-dimensional hover coordinate. At time t₂, if the firstpointer image Ip₁ in the first image IF₁ and the second pointer imageIp₂ in the second image IF₂ have different areas or widths, it meansthat the finger of user is moved forward (i.e. along the Z-axis) suchthat the first image sensor 111 detects the finger image but the secondimage sensor 12 detects a partial image of the first or palm. In thiscase, it is able to identify that a click event occurs at the time t₂.In this embodiment, when the processing unit 12 identifies that a sizedifference of the pointer images between the first image IF₁ and thesecond image IF₂ captured at the same time (i.e. the same samplingperiod) is larger than or equal to a difference threshold, a click eventalong the Z-axis is identified.

In addition, in this aspect the processing unit 12 identifies the sizedifference of the pointer images between the image frames captured atthe same sampling period by different image sensors, and the purpose isto distinguish the difference between finger image from first image orpalm image. However, when the first image sensor 111 and the secondimage sensor 112 detect images of other parts of human body (e.g. awrist image), the first pointer image Ip₁ in the first image IF₁ and thesecond pointer image Ip₂ in the second image IF₂ may also havesubstantially identical widths or areas. In order to avoid thissituation, when sizes of the pointer images in the first image IF₁ andthe second image IF₂ are larger than a predetermined size (e.g. an upperlimit may be set according to actual finger images), the processing unitmay not activate so as to avoid misoperation; that is, the X-axiscoordinate and the Y-axis coordinate are not calculated and the clickevent is not identified.

In addition, in order to eliminate the interference from ambient light,the light source 13 may emit light with difference brightness valuescorresponding to the image capturing of the first image sensor 111 andthe second image sensor 112. For example referring to FIG. 12, the lightsource 13 emits light alternative in a first brightness value B₁ and asecond brightness value B₂ corresponding to the image capturing of theimage sensors. The first image sensor 111 and the second image sensor112 respectively capture a bright image IF_(B) corresponding to thefirst brightness value B₁ and respectively capture a dark image IF_(D)corresponding to the second brightness value B₂. The processing unit 12further calculates a differential image (IF_(B)−IF_(D)) between thebright image IF_(B) and the dark image IF_(D) captured by each imagesensor so as to eliminate the interference from ambient light, whereinthe second brightness value B₂ may be zero or nonzero brightness. Inother words, in the above aspect, the first image IF₁ and the secondimage IF₂ may both be differential images. In this aspect, the brightimage is referred to an image frame captured when the light source 13turns on and the dark image is referred to an image frame captured whenthe light source 13 turns off or emits light in a lower brightnessvalue.

Referring to FIGS. 2A, 2B and 6, FIG. 6 shows an upper view of a secondaspect of the gesture detection device 1 according to the firstembodiment of the present disclosure. In this aspect, the first imagesensor 111 is disposed on the detection surface 10 and configured tocapture first images IF₁ substantially along the normal direction “n”;the second image sensor 112 is disposed on the detection surface 10parallel to the first image sensor 111 (i.e. no position offset alongthe Z-axis) and configured to capture second images IF₂ substantiallyalong the normal direction “n”; and the processing unit 12 is configuredto calculate an X-axis coordinate according to a position of a pointerimage in the first images IF₁ or the second images IF₂, calculate aY-axis coordinate according to a disparity of pointer images in thefirst images IF₁ and the second images IF₂, and identify a click eventwhen a size variation of the pointer images between the first images IF₁captured at different times and/or between the second images IF₂captured at different times is larger than or equal to a variationthreshold. The difference between the second aspect and the first aspectis that in the second aspect there is no position offset between thefirst image sensor 111 and the second image sensor along the Z-axis. Inthis aspect, methods of calculating the X-axis coordinate and the Y-axiscoordinate are identical to those of the first aspect (referring toFIGS. 8 and 9) and thus details thereof are not repeated herein. Themethod of identifying the click event is illustrated hereinafter.

Referring to FIG. 10B, in this aspect the first image sensor 111 and thesecond image sensor 112 are also configured to detect a hoveringoperation of a finger and a click event. For example at time t₁, if thefirst pointer image Ip₁ in the first image IF₁ and the second pointerimage Ip₂ in the second image IF₂ have substantially identical areas orwidths, it means that it is the finger of user between the first imagesensor 111 and the second image 112, and the user is performing thehovering operation now. In hovering operation the X-axis coordinate andY-axis coordinate may be calculated by the method of FIGS. 8 and 9 to beserved as the two-dimensional hover coordinate. At time t₂, if the areasor widths of the first pointer image Ip₁′ in the first image IF₁ and thesecond pointer image Ip₂ in the second image IF₂′ have a change, itmeans that the finger of user is moved forward such that the first imagesensor 111 and/or the second image sensor 12 detects a partial image ofthe first or palm, and it is able to identify that a click event occursat the time t₂. In this embodiment, when the processing unit 12identifies a size variation (e.g. the size variation between Ip₁ andIp₁′ and/or the size variation between Ip₂ and Ip₂′) of the pointerimages between the first images IF₁ captured at different times (i.e.different sampling times, e.g. t₁ and t₂) and/or between the secondimages IF₂ captured at different times is larger than or equal to avariation threshold, a click event along the Z-axis is identified,wherein the size variation associated with the first images IF₁ and thesecond images IF₂ may be different. In this aspect, the times t₁ and t₂are preferably two adjacent sampling times.

As mentioned above, in order to avoid misoperation, in this aspect whensizes of the pointer images in the first image IF₁ and the second imageIF₂ are larger than a predetermined size, the processing unit 12 doesnot activate.

Similarly, in order to eliminate the influence of ambient light, in thisaspect the light source 13 may emit light of different brightness valuesalternatively corresponding to the image capturing of the image sensors111 and 112 as shown in FIG. 12 to allow the processing unit 12 tocalculate the differential image.

Referring to FIGS. 2A, 2B and 7A, FIG. 7A shows an upper view of a thirdaspect of the gesture detection device 1 according to the firstembodiment of the present disclosure. This aspect includes a first imagesensor 111, a second image sensor 112 and a third image senor 113,wherein the first image sensor 111 is disposed on the detection surface10 and configured to capture a first image IF₁ substantially along thenormal direction “n”; the second image sensor 112 is disposed on thedetection surface 10 parallel to the first image sensor 111 (i.e. noposition offset along the Z-axis) and configured to capture a secondimage IF₂ substantially along the normal direction “n”; and the thirdimage sensor 113 is disposed on the detection surface 10 offsetting apredetermined distance D from the first image sensor 111 and the secondimage sensor along a Z-axis and configured to capture a third image IF₃substantially along the normal direction “n”. In this aspect, theprocessing unit 12 is configured to calculate a two-dimensionalcoordinate according to the first image IF₁ and the second image IF₂;i.e. the X-axis coordinate and Y-axis coordinate, wherein the method ofcalculating the two-dimensional coordinate is identical to that of thefirst aspect (referring to FIGS. 8 and 9) and thus details thereof arenot repeated herein. In addition, the processing unit 12 is furtherconfigured to identify a click event along the Z-axis when a sizedifference between a pointer image in the first image IF₁ and a pointerimage in the third image IF₃ or between a pointer image in the secondimage IF₂ and the pointer image in the third image IF₃ is larger than orequal to a difference threshold, wherein the method of identifying aclick event is identical to that of the first aspect (referring to FIG.10A) and thus details thereof are not repeated herein.

More specifically speaking, in the third aspect the processing unit 12calculates the two-dimensional coordinate according to the two imagesensors disposed in parallel (similar to FIG. 6) and identifies theclick event according to the two image sensors having a position offsetalong the Z-axis (similar to FIG. 5). However, the arrangement of theimage sensors is not limited to FIG. 7A, the image sensors may bearranged in the way as shown in FIGS. 7B and 7C, wherein in FIGS. 7A-7Cthe hover coordinate is calculated according to the image framescaptured by the first image sensor 111 and the second image sensor 112.In addition in FIG. 7B, the processing unit 12 identifies the clickevent according to the image frames captured by the second image sensor112 and the third image sensor 113. In FIG. 7C, the processing unit 12identifies the click event according to the image frames captured by thefirst image sensor 11 and the third image sensor 113.

As mentioned above, in order to avoid misoperation, in this aspect whensizes of the pointer images in the first image IF₁ and the second imageIF₂ are larger than a predetermined size, the processing unit 12 doesnot activate.

Similarly, in order to eliminate the influence of ambient light, in thisaspect the light source 13 may emit light of different brightness valuesalternatively corresponding to the image capturing of the image sensors111 to 113 as shown in FIG. 12 to allow the processing unit 12 tocalculate the differential image.

Referring to FIGS. 3A, 3B and 5, FIG. 5 shows an upper view of a firstaspect of the gesture detection device 1 according to the secondembodiment of the present disclosure. In this aspect, the first imagesensor 111 is disposed on the detection surface 10 and configured tocapture a first image IF₁ toward a direction deviating a predeterminedangle (e.g. +θ) from the normal direction “n”; the second image sensor112 is disposed on the detection surface 10 offsetting a predetermineddistance D from the first image sensor 111 along a Z-axis and configuredto capture a second image IF₂ toward a direction deviating a negativepredetermined angle (e.g. −θ) from the normal direction “n”; and theprocessing unit 12 is configured to calculate a two-dimensionalcoordinate according to positions of pointer images in the first imageIF₁ and the second image IF₂, and identify a click event along theZ-axis when a size difference of the pointer images between the firstimage IF₁ and the second image IF₂ is larger than or equal to adifference threshold. The difference between this aspect and the firstaspect of the first embodiment is that in this aspect the image sensorsincline a predetermined angle (e.g. θ) toward each other to captureimages such that the pointer may be within field of views of all imagesensors when the image sensors are separated by a larger distance. Inthis aspect, the method of identifying the click event is identical tothat of the first aspect of the first embodiment (referring to FIG.10A); i.e. when the processing unit 12 identifies that a size difference(e.g. the difference between Ip₂′ and Ip₁′) of the pointer imagesbetween the first image IF₁ and the second image IF₂ captured at thesame time is larger than or equal to a difference threshold, a clickevent along the Z-axis is identified. The method of calculating thetwo-dimensional coordinate is illustrated hereinafter.

Referring to FIG. 11, in this aspect the X-axis coordinate and theY-axis coordinate may be obtained simultaneously according to the firstimage IF₁ and the second image IF₂. For example, each position of thefirst image IF₁ and the second image IF₂ corresponds to a relative angleof the pointer position. Accordingly, when a pointer enters field ofviews of the first image sensor 111 and the second image sensor 112, theprocessing unit 12 may calculate a pointer image position (e.g. thecenter or gravity center) of pointer images in the first image 111 andthe second image 112, and each pointer image position corresponds to oneangle, e.g. 30 and 60 degrees shown herein. Meanwhile, if one of thefirst image sensor 111 and the second image sensor 112 is set as theoriginal point, i.e. having a plane coordinate (0,0), two linearequations , e.g. L₁ and L₂ may be determined. The processing unit 12 maycalculate the cross point coordinate of the two linear equations L₁ andL₂ to be served as the two-dimensional coordinate of the pointerposition. In another embodiment, a lookup table of two-dimensionalcoordinates versus two angles associated with pointer image positionsmay be constructed previously, and the processing unit 12 may determinethe two-dimensional coordinate according to the pointer image positionsin the first image IF₁ and the second image IF₂ as well as the lookuptable.

As mentioned above, in order to avoid misoperation, in this aspect whensizes of the pointer images in the first image IF₁ and the second imageIF₂ are larger than a predetermined size, the processing unit 12 doesnot activate.

Similarly, in order to eliminate the influence of ambient light, in thisaspect the light source 13 may emit light of different brightness valuesalternatively corresponding to the image capturing of the image sensors111 and 112 as shown in FIG. 12 to allow the processing unit 12 tocalculate the differential image.

Referring to FIGS. 3A, 3B and 6, FIG. 6 shows an upper view of a secondaspect of the gesture detection device 1 according to the secondembodiment of the present disclosure. In this aspect, the first imagesensor 111 is disposed on the detection surface 10 and configured tocapture first images IF₁ toward a direction deviating a predeterminedangle (e.g. +θ) from the normal direction “a”; the second image sensor112 is disposed on the detection surface 10 parallel to the first imagesensor 111 (i.e. no position offset along the Z-axis) and configured tocapture second images IF₂ toward a direction deviating a negativepredetermined angle (e.g. −θ) from the normal direction “n”; and theprocessing unit 12 is configured to calculate a two-dimensionalcoordinate according to positions of pointer images in the first imagesIF₁ and the second images IF₂, and identify a click event when a sizevariation of the pointer images between the first images IF₁ captured atdifferent times (i.e. different sampling times) and between the secondimages IF₂ captured at different times is larger than or equal to avariation threshold. The difference between this aspect and the secondaspect of the first embodiment is that in this aspect the image sensorsincline a predetermined angle (e.g. θ) toward each other to captureimages such that the pointer may be within field of views of all imagesensors when the image sensors are separated by a larger distance. Inthis aspect, the method of identifying the click event is identical tothat of the second aspect of the first embodiment (referring to FIG.10B); i.e. when the processing unit 12 identifies a size variation (e.g.the size variation between Ip₁ and 1p₁′ and/or the size variationbetween Ip₂ and Ip₂′) of the pointer images between the first images IF₁captured at different times and/or between the second images IF₂captured at different times is larger than or equal to a variationthreshold, a click event along the Z-axis is identified. The method ofcalculating the two-dimensional coordinate is identical to that of thefirst aspect of the second embodiment (referring to FIG. 11) and thusdetails thereof are not repeated herein.

Briefly speaking, the processing unit 12 identifies the click eventaccording to the variation of the pointer images between the imageframes captured at different sampling times by the image sensors, anddetermines the two-dimensional coordinate according to two linearequations formed according to positions of the pointer images in thefirst image IF₁ and the second image IF₂ or according to a lookup tableand positions of the pointer images in the first image IF₁ and thesecond image IF₂.

As mentioned above, in order to avoid misoperation, in this aspect whensizes of the pointer images in the first image IF₁ and the second imageIF₂ are larger than a predetermined size, the processing unit 12 doesnot activate.

Similarly, in order to eliminate the influence of ambient light, in thisaspect the light source 13 may emit light of different brightness valuesalternatively corresponding to the image capturing of the image sensors111 and 112 as shown in FIG. 12 to allow the processing unit 12 tocalculate the differential image.

Referring to FIGS. 3A, 3B and 7A, FIG. 7A shows an upper view of a thirdaspect of the gesture detection device 1 according to the secondembodiment of the present disclosure. This aspect includes a first imagesensor 111, a second image sensor 112 and a third image senor 113,wherein the first image sensor 111 is disposed on the detection surface10 and configured to capture a first image IF₁ toward a directiondeviating a predetermined angle (e.g. +θ) from the normal direction “n”;the second image sensor 112 is disposed on the detection surface 10parallel to the first image sensor 111 (i.e. no position offset alongthe Z-axis) and configured to capture a second image IF₂ toward adirection deviating a negative predetermined angle (e.g. −θ) from thenormal direction “n”; and the third image sensor 113 is disposed on thedetection surface 10 offsetting a predetermined distance D from thefirst image sensor 111 and the second image sensor 112 along a Z-axisand configured to capture a third image IF₃ substantially along thenormal direction “n”. In this aspect, the processing unit 12 isconfigured to calculate a two-dimensional coordinate according to thefirst image IF₁ and the second image IF₂; i.e. the X-axis coordinate andY-axis coordinate, wherein the method of calculating the two-dimensionalcoordinate is identical to that of the first aspect of the secondembodiment (referring to FIG. 11) and thus details thereof are notrepeated herein. In addition, the processing unit 12 is furtherconfigured to identify a click event along the Z-axis when a sizedifference between a pointer image in the first image IF₁ and a pointerimage in the third image IF₃ or between a pointer image in the secondimage IF₂ and the pointer image in the third image IF₃ is larger than orequal to a difference threshold, wherein the method of identifying aclick event is identical to that of the first aspect of the firstembodiment (referring to FIG. 10A) and thus details thereof are notrepeated herein. In addition, a deviation direction of the center lineof the field of view of the third image sensor 113 with respect to thenormal line may be determined according to the disposed position of thethird image sensor 113. For example, in FIG. 7B the third image sensor113 may rotate toward the right hand side (e.g. +θ) whereas in FIG. 7Cthe third image sensor 113 may rotate toward the left hand side (e.g.−θ).

Briefly speaking, in this aspect the processing unit 12 identifies theclick event according to the difference between pointer images in theimage frames captured at the same sampling time by the two image sensorshaving a position offset in the Z-axis, and determines thetwo-dimensional coordinate according to two linear equations formedaccording to positions of the pointer images in the two image sensorsdisposed in parallel or according to a lookup table and positions of thepointer images in the two image sensors disposed in parallel.

As mentioned above, in order to avoid misoperation, in this aspect whensizes of the pointer images in the first image IF₁ and the second imageIF₂ are larger than a predetermined size, the processing unit 12 doesnot activate.

Similarly, in order to eliminate the influence of ambient light, in thisaspect the light source 13 may emit light of different brightness valuesalternatively corresponding to the image capturing of the image sensors111 to 113 as shown in FIG. 12 to allow the processing unit 12 tocalculate the differential image.

In every aspect of the present disclosure, the at least two imagesensors (e.g. 111-113) preferably capture image frames synchronously. Itshould be mentioned that the XYZ-axes shown in FIGS. 5-7C are onlyintended to illustrate but not to limit the present disclosure.

As mentioned above, it is difficult for the conventional optical gesturerecognition system to identify the click event so that the applicablefunctions are limited. Therefore, the present disclosure furtherprovides a gesture detection device (FIGS. 1, 2A-2B and 3A-3B) that mayperform both the hovering operation and the click operation andeliminate the environmental interference by calculating the differentialimage.

Although the disclosure has been explained in relation to its preferredembodiment, it is not used to limit the disclosure. It is to beunderstood that many other possible modifications and variations can bemade by those skilled in the art without departing from the spirit andscope of the disclosure as hereinafter claimed.

What is claimed is:
 1. A gesture detection device, comprising: a detection surface having a normal direction; a first image sensor disposed on the detection surface and capturing a first image along the normal direction; a second image sensor disposed on the detection surface offsetting a predetermined distance from the first image sensor along a Z-axis and capturing a second image along the normal direction; and a processing unit configured to calculate an X-axis coordinate according to a position of a pointer image in the first image or a pointer image in the second image, calculate a Y-axis coordinate according to a disparity of the pointer images in the first image and the second image, and identify a click event of the Z-axis when a size difference of the pointer images between the first image and the second image is larger than or equal to a difference threshold.
 2. The gesture detection device as claimed in claim 1, wherein when sizes of the pointer images in the first image and the second image are larger than or equal to a predetermined size, the processing unit stops calculating the X-axis coordinate and the Y-axis coordinate and stops identifying the click event.
 3. The gesture detection device as claimed in claim 1, further comprising an infrared light source configured to illuminate field of views of the first image sensor and the second image sensor.
 4. The gesture detection device as claimed in claim 3, wherein the infrared light source emits light alternatively in a first brightness value and a second brightness value, and the first image sensor and the second image sensor respectively capture a bright image corresponding to the first brightness value and respectively capture a dark image corresponding to the second brightness value, and the processing unit further calculates a differential image between the bright image and the dark image.
 5. The gesture detection device as claimed in claim 4, wherein the second brightness value is zero or nonzero brightness.
 6. The gesture detection device as claimed in claim 1, wherein the first image sensor and the second image sensor comprise a linear image sensor array.
 7. A gesture detection device, comprising: a detection surface having a normal direction; a first image sensor disposed on the detection surface and capturing first images along the normal direction; a second image sensor disposed on the detection surface parallel to the first image sensor and capturing second images along the normal direction; and a processing unit configured to calculate an X-axis coordinate according to a position of a pointer image in the first images or a pointer image in the second images, calculate a Y-axis coordinate according to a disparity of the pointer images in the first images and the second images, and identify a click event when a size variation of the pointer images between the first images captured at different times and/or between the second images captured at different times is larger than or equal to a variation threshold.
 8. The gesture detection device as claimed in claim 7, wherein when sizes of the pointer images in the first images and the second images are larger than or equal to a predetermined size, the processing unit stops calculating the X-axis coordinate and the Y-axis coordinate and stops identifying the click event.
 9. The gesture detection device as claimed in claim 7, further comprising an infrared light source configured to illuminate field of views of the first image sensor and the second image sensor.
 10. The gesture detection device as claimed in claim 9, wherein the infrared light source emits light alternatively in a first brightness value and a second brightness value, and the first image sensor and the second image sensor respectively capture a bright image corresponding to the first brightness value and respectively capture a dark image corresponding to the second brightness value, and the processing unit further calculates a differential image between the bright image and the dark image.
 11. The gesture detection device as claimed in claim 10, wherein the second brightness value is zero or nonzero brightness.
 12. The gesture detection device as claimed in claim 7, wherein the first image sensor and the second image sensor comprise a linear image sensor array.
 13. A gesture detection device, comprising: a detection surface having a normal direction; a first image sensor disposed on the detection surface and capturing a first image along the normal direction; a second image sensor disposed on the detection surface parallel to the first image sensor and capturing a second image along the normal direction; a third image sensor disposed on the detection surface offsetting a predetermined distance from the first image sensor and the second image sensor along a Z-axis and capturing a third image along the normal direction; and a processing unit configured to calculate a two-dimensional coordinate according to the first image and the second image, and identify a click event of the Z-axis when a size difference between a pointer image in the first image and a pointer image in the third image or between a pointer image in the second image and the pointer image in the third image is larger than or equal to a difference threshold.
 14. The gesture detection device as claimed in claim 13, wherein the two-dimensional coordinate comprises an X-axis coordinate and a Y-axis coordinate, and the processing unit calculates an X-axis coordinate according to a position of the pointer image in the first image or the pointer image in the second image, and calculates a Y-axis coordinate according to a disparity of the pointer images in the first image and the second image.
 15. The gesture detection device as claimed in claim 13, wherein when sizes of the pointer images in the first image and the second image are larger than or equal to a predetermined size, the processing unit stops calculating the two-dimensional coordinate and stops identifying the click event.
 16. The gesture detection device as claimed in claim 13, further comprising an infrared light source configured to illuminate field of views of the first image sensor, the second image sensor and the third image sensor.
 17. The gesture detection device as claimed in claim 16, wherein the infrared light source emits light alternatively in a first brightness value and a second brightness value, and the first image sensor, the second image sensor and the third image sensor respectively capture a bright image corresponding to the first brightness value and respectively capture a dark image corresponding to the second brightness value, and the processing unit further calculates a differential image between the bright image and the dark image.
 18. The gesture detection device as claimed in claim 17, wherein the second brightness value is zero or nonzero brightness.
 19. The gesture detection device as claimed in claim 13, wherein the first image sensor, the second image sensor and the third image sensor comprise a linear image sensor array. 